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WHAT IS CLAIMED IS: 

1. A background replacing apparatus comprising: 
an image obtaining section that obtains a 

5 plurality of photographic images generated through 

photography of a common subject under a plurality of 
photographic conditions ; 

an area discriminating section that discriminates 
among a subject area, a background area, and a boundary 
10 area of the photographic image in accordance with at least 
one of the photographic images obtained by the image 
obtaining section; 

a mixing ratio determining section that determines 
a mixing ratio of a subject color to a background color in 
15 the boundary area; and 

a background replacing section that replaces a 
background of the photographic image obtained by the image 
obtaining section with a different background in accordance 
with three areas of the subject area, the background area, 
20 and the boundary area, which are discriminated by the area 
discriminating section, and the mixing ratio determined by 
the mixing ratio determining section. 

2 . A background replacing apparatus according to 
25 claim 1, wherein the area discriminating section 

discriminates among the subject area, the background area, 
and the boundary area of the photographic image in 
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accordance with a predetermined different point between the 
subject and the background in the photographic conditions. 

3. A background replacing apparatus according to 
5 claim 1, wherein the area discriminating section 

discriminates the photographic image between a subject-side 
area and a background-side area in accordance with a 
predetermined different point between the subject and the 
background in the photographic conditions, and at least one 
10 of the subject-side area and the background-side area is 
reduced so as to be treated as the subject area and the 
background area, and an area between the subject area and 
the background area is treated as the boundary area. 

15 4. A background replacing apparatus according to 

claim 1, wherein the area discriminating section 
discriminates the photographic image between a subject-side 
area and a background-side area in accordance with a 
predetermined different point between the subject and the 

20 background in the photographic conditions, and the subject- 
side area is reduced so as to be treated as the subject 
area, and an area portion excepting the subject area, of 
the subject-side area before reduction, is treated as whole 
or part of the boundary area. 

25 

5. A background replacing apparatus according to 
claim 1, wherein the area discriminating section 
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discriminates the photographic image between a subject-side 
area and a background-side area in accordance with a 
predetermined different point between the subject and the 
background in the photographic conditions, and the 
background-side area is reduced so as to be treated as the 
background area, and an area portion excepting the 
background area, of the background-side area before 
reduction, is treated as whole or part of the boundary area. 

6. A background replacing apparatus according to 
claim 1, wherein the area discriminating section 
discriminates among the subject area, the background area, 
and the boundary area of the photographic image in 
accordance with one of the photographic images obtained by 
the image obtaining section, and 

the background replacing section replaces a 
background of a photographic image, which is different from 
the photographic image used for discrimination of areas by 
the area discriminating section, of the photographic images 
obtained by the image obtaining section. 

7. A background replacing apparatus according to 
claim 1, wherein the mixing ratio determining section 
presumes the subject color and the mixing ratio in the 
boundary area of the photographic image; and 

the background replacing section replaces the 
background of the photographic image with the different 



background using the subject color and 
which are presumed by the mixing ratio 



the mixing ratio, 
determining section. 



8. A background replacing apparatus according to 
claim 7, wherein the mixing ratio determining section 
presumes the subject color using colors in the subject an 
as candidates for the subject color. 



9. A background replacing apparatus according to 
claim 7, wherein the mixing ratio determining section 
presumes the subject color and the mixing ratio on 
individual points in the boundary area, and presumes the 
subject color on assumption that the subject color lines up 
on a straight line basis with the background color and a 
color of the associated point in a predetermined color 
space . 



10. A background replacing apparatus according to 
claim 7, wherein the mixing ratio determining section 
presumes the subject color and the mixing ratio on 
individual points in the boundary area, and presumes the 
subject color, using colors in the subject area as 
candidates for the subject color, in such a manner that of 
the candidates, a candidate, which lines up on a straight 
line basis with the background color and a color of the 
associated point in a predetermined color space, and is 
closest to the color of the associated point, is presumed 
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as the subject color. 



11. A background replacing apparatus according to 
claim 7, wherein the mixing ratio determining section 
presumes the subject color and the mixing ratio on 
individual points in the boundary area, and presumes the 
mixing ratio in accordance with a ratio of a mutual 
distance among the background color, a color of the 
associated point, and the subject color, in a predetermined 
color space. 



12. A background replacing apparatus according to 
claim 7, wherein the background replacing section replaces 
the background of the photographic image with the different 
background in such a manner that a color of the different 
background and the subject color presumed by the mixing 
ratio determining section are mixed at the mixing ratio 
presumed by the mixing ratio determining section. 



13. A background replacing apparatus according to 
claim 1, wherein the background replacing apparatus further 
comprises an image correcting section performs correction 
of the area discriminated by the area discriminating 
section in response to an operation, and causes the 
background replacing section to perform the background 
replacing based on a corrected area so that a background 
replaced image is corrected. 
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14 . A background replacing apparatus according to 
claim 13, wherein the area discriminating section 
discriminates the photographic image between a subject-side 

5 area and a background-side area in accordance with a 

predetermined different point between the subject and the 
background in the photographic conditions, and the subject- 
side area and the background-side area are reduced by a 
predetermined reduction amount so as to be treated as the 
10 subject area and the background area, and 

the image correcting section alters the reduction 
amount on the area discriminating section in response to an 
operation, so that the subject area and/or the background 
area are corrected. 

15 

15. A background replacing apparatus according to 
claim 13, wherein the area discriminating section 
discriminates between the subject area and the background 
area in accordance with a discrimination basis based on a 

20 predetermined different point between the subject and the 
background in the photographic conditions, and 

the image correcting section alters the 
discrimination basis in response to an operation, so that 
the subject area and/or the background area are corrected. 

25 

16. A background replacing apparatus comprising: 
an image obtaining section that obtains a 

164 



plurality of photographic images generated through 
photography of a common subject under a plurality of 
photographic conditions ; 

an area discriminating section that discriminates 
a background area of the photographic image from other 
areas in accordance with at least one of the photographic 
images obtained by the image obtaining section; 

a mixing state presuming section that presumes a 
mixing ratio of a subject color and a background color in 
other areas excepting the background area in the 
photographic image, and the subject color; and 

a background replacing section that replaces a 
background of the photographic image obtained by the image 
obtaining section with a different background using the 
subject color and the mixing ratio, which are presumed by 
the mixing state presuming section. 



17. A background replacing apparatus according to 

claim 16, wherein the mixing state presuming section 

presumes the subject color using colors in the subject area 

as candidates for the subject color. 

» 

18. A background replacing apparatus according to 
claim 16, wherein the mixing state presuming section 
presumes the subject color and the mixing ratio on 
individual points in other areas excepting the background 
area, and presumes the subject color on assumption that the 



subject color lines up on a straight line basis with the 
background color and a color of the associated point in a 
predetermined color space. 

19. A background replacing apparatus according to 
claim 16, wherein the mixing state presuming section 
presumes the subject color and the mixing ratio on 
individual points in other areas excepting the background 
area, and presumes the subject color, using colors in other 
areas excepting the background area as candidates for the 
subject color, in such a manner that of the candidates, a 
candidate, which lines up on a straight line basis with the 
background color and a color of the associated point in a 
predetermined color space, and is farthest from the color 
of the associated point, is presumed as the subject color. 

20. A background replacing apparatus according to 
claim 16, wherein the mixing state presuming section 
presumes the subject color and the mixing ratio on 
individual points in other areas excepting the background 
area, and presumes the mixing ratio in accordance with a 
ratio of a mutual distance among the background color, a 
color of the associated point, and the subject color, in a 
predetermined color space. 

21. A background replacing apparatus according to 
claim 16, wherein the background replacing section replaces 



the background of the photographic image with the different 
background in such a manner that a color of the different 
background and the subject color presumed by the mixing 
state presuming section are mixed at the mixing ratio 
presumed by the mixing state presuming section. 

22. A background replacing apparatus according to 
claim 16, wherein the area discriminating section 
discriminates a background area of the photographic image 
from other areas in accordance with a ratio of luminous 
intensity among the plural photographic images. 

23. A background replacing apparatus according to 
claim 16, further comprising a storage section that stores 
the subject color and the mixing ratio, which are presumed 
by the mixing state presuming section, 

wherein the background replacing section replaces 
a background of the photographic image obtained by the 
image obtaining section with a different background using 
the subject color and the mixing ratio, which are stored in 
the storage section. 

24. A background replacing apparatus according to 
claim 16, further comprising a first parameter determining 
section that determines a correction parameter to be used 
for a color correction for the subject color in accordance 
with an image of said other area in which a color is 
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replaced by the subject color presumed in the mixing state 
presuming section, and 

a first color correcting section that applies the 
color correction to the subject color, which is presumed by 
the mixing state presuming section, using the correction 
parameter determined by the first parameter determining 
section, 

wherein the background replacing section replaces 
the background of the photographic image obtained by the 
image obtaining section with a different background using 
the subject color subjected to the color correction by the 
first color correcting section and the mixing ratio. 



25. A background replacing apparatus according to 
claim 16, further comprising a second parameter determining 
section that determines a correction parameter to be used 
for a color correction for the subject color in accordance 
with the photographic image obtained by the image obtaining 
section, and 

a second color correcting section that applies the 
color correction to the subject color, which is presumed by 
the mixing state presuming section, using the correction 
parameter determined by the second parameter determining 
section, 

wherein the background replacing section replaces 
the background of the photographic image obtained by the 
image obtaining section with a different background using 
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the subject color subjected to the color correction by the 
second color correcting section and the mixing ratio. 



26. A background replacing apparatus according to 
claim 16, further comprising a third parameter determining 
section that determines a correction parameter to be used 
for a color correction for the subject color to a parameter 
according to an operation, and 

a third color correcting section that applies the 
color correction to the subject color, which is presumed by 
the mixing state presuming section, using the correction 
parameter determined by the third parameter determining 
section, 

wherein the background replacing section replaces 
the background of the photographic image obtained by the 
image obtaining section with a different background using 
the subject color subjected to the color correction by the 
third color correcting section and the mixing ratio. 



27. A background replacing apparatus according to 
claim 16, further comprising a fourth parameter determining 
section that determines a correction parameter to be used 
for a color correction for the subject color to a parameter 
according to the different background, and 

a fourth color correcting section that applies the 
color correction to the subject color, which is presumed by 
the mixing state presuming section, using the correction 
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parameter determined by the fourth parameter determining 
section, 

wherein the background replacing section replaces 
the background of the photographic image obtained by the 
image obtaining section with the different background using 
the subject color subjected to the color correction by the 
fourth color correcting section and the mixing ratio. 



28. A background replacing apparatus according to 
claim 16, wherein the background replacing section replaces 
a background of the photographic image using, as the 
different background, a background to which relation 
information representative of a relative positional 
relation in an image between the subject and the background 
is applied, and performs an replacement in such a manner 
that the relative relation in the background replaced image 
between the subject and the background is the same relative 
relation as the relative relation represented by the 
relation information . 



29. A background replacing apparatus comprising: 
an image obtaining section that obtains a 
plurality of photographic images generated through 
photography of a common subject under a plurality of 
photographic conditions mutually different in color of the 
background; 

an area discriminating section that discriminates 
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a background area of the photographic image from other 
areas in accordance with the photographic images obtained 
by the image obtaining section; 

a mixing state presuming section that presumes a 
mixing ratio of the subject color to the background color, 
and the subject color, in other areas excepting the 
background area in the photographic image, in accordance 
with the plurality of photographic images; and 

a background replacing section that replaces a 
background of the photographic image obtained by the image 
obtaining section with a different background using the 
subject color and the mixing ratio, which are presumed by 
the mixing state presuming section. 

30. A background replacing apparatus according to 
claim 29, wherein the mixing state presuming section 
presumes, on assumption that the plurality of photographic 
images are equal to each other in terms of mixing state of 
the subject color and the background color, the subject 
color and the mixing ratio. 

31. A background replacing apparatus according to 
claim 19, wherein the mixing state presuming section 
presumes the subject color and the mixing ratio on 
individual points in other area excepting the background 
area, determines straight lines each coupling in a 
predetermined color space the background color with a color 



of the associated point in each of the plurality of 
photographic images generated through photography of the 
common subject under a plurality of photographic conditions 
mutually different in color of the background, and presumes 
5 a color corresponding to an intersection of the straight 
lines as the subject color. 

32. A background replacing apparatus according to 
claim 29, wherein the image obtaining section obtains a 
10 plurality of photographic images generated through 

photography of the common subject under a plurality of 
photographic conditions mutually different in color-phase 
of the background. 

15 33. A background replacing apparatus according to 

claim 29, wherein the area discriminating section 
discriminates between a subject area and a boundary area of 
other areas excepting the background area in the 
photographic image; and 

20 the mixing state presuming section that presumes 

the subject color and the mixing ratio in the boundary area. 

34. A background replacing apparatus according to 
claim 29, wherein the area discriminating section 
25 discriminates the background area of the photographic image 
obtained by the image obtaining section from other areas in 
accordance with an amount of the change in color among the 
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plurality of photographic images. 

35. A background replacing apparatus according to 
claim 29, wherein the area discriminating section 
5 discriminates the background area of the photographic image 
obtained by the image obtaining section from other areas in 
accordance with a difference of luminous intensity between 
the background and the subject. 

10 36. A background replacing program that causes a 

computer to operate as an apparatus, when the background 

replacing program is incorporated into the computer and is 

executed, the apparatus comprising: 

an image obtaining section that obtains a 
15 plurality of photographic images generated through 

photography of a common subject under a plurality of 

photographic conditions ; 

an area discriminating section that discriminates 

among a subject area, a background area, and a boundary 
20 area of the photographic image in accordance with the 

photographic images obtained by the image obtaining 

section; 

a mixing ratio determining section that determines 
a mixing ratio of a subject color to a background color in 
25 the boundary area; and 

a background replacing section that replaces a 
background of the photographic image obtained by the image 



173 



obtaining section with a different background in accordance 
with three areas of the subject area, the background area, 
and the boundary area, which are discriminated by the area 
discriminating section, and the mixing ratio determined by 
the mixing ratio determining section. 



37. A background replacing program that causes a 
computer to operate as an apparatus, when the background 
replacing program is incorporated into the computer and is 

10 executed, the apparatus comprising: 

an image obtaining section that obtains a 
plurality of photographic images generated through 
photography of a common subject under a plurality of 
photographic conditions; 

15 an area discriminating section that discriminates 

a background area of the photographic image from other 
areas in accordance with the photographic images obtained 
by the image obtaining section; 

a mixing state presuming section that presumes a 

20 mixing ratio of the subject color and the background color, 
and the subject color, in other areas excepting the 
background area in the photographic image; and 

a background replacing section that replaces a 
background of the photographic image obtained by the image 

25 obtaining section with a different background using the 

subject color and the mixing ratio, which are presumed by 
the mixing state presuming section. 
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38. A background replacing program that causes a 
computer to operate as an apparatus, when the background 
replacing program is incorporated into the computer and is 
5 executed, the apparatus comprising: 

an image obtaining section that obtains a 
plurality of photographic images generated through 
photography of a common subject under a plurality of 
photographic conditions in which colors of backgrounds are 

10 different from one another; 

an area discriminating section that discriminates 
a background area of the photographic image from other 
areas in accordance with the photographic images obtained 
by the image obtaining section; 

15 a mixing state presuming section that presumes a 

mixing ratio of the subject color and the background color, 
and the subject color, in other areas excepting the 
background area in the photographic image, in accordance 
with the plurality of photographic images; and 

20 a background replacing section that replaces a 

background of the photographic image obtained by the image 
obtaining section with a different background using the 
subject color and the mixing ratio, which are presumed by 
the mixing state presuming section. 

25 

39. A background replacing method comprising: 
an image obtaining step that obtains a plurality 
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of photographic images generated through photography of a 
common subject under a plurality of photographic 
conditions ; 

an area discriminating step that discriminates 
5 among a subject area, a background area, and a boundary 
area of the photographic image in accordance with the 
photographic images obtained by the image obtaining step; 

a mixing ratio determining step that determines a 
mixing ratio of a subject color to a background color in 
10 the boundary area; and 

a background replacing step that replaces a 
background of the photographic image obtained by the image 
obtaining step with a different background in accordance 
with three areas of the subject area, the background area, 
15 and the boundary area, which are discriminated by the area 
discriminating step, and the mixing ratio determined by the 
mixing ratio determining step. 

.40. A background replacing method comprising: 
20 an image obtaining step that obtains a plurality 

of photographic images generated through photography of a 
common subject under a plurality of photographic 
conditions ; 

an area discriminating step that discriminates a 
25 background area of the . photographic image from other areas 
in accordance with the photographic images obtained by the 
image obtaining step; 
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a mixing state presuming step that presumes a 
mixing ratio of a subject color and a background color, and 
the subject color, in other areas excepting the background 
area in the photographic image; and 
5 a background replacing step that replaces a 

background of the photographic image obtained by the image 
obtaining step with a different background using the 
subject color and the mixing ratio, which are presumed by 
the mixing state presuming step. 

10 

41. A background replacing method comprising: 
an image obtaining step that obtains a plurality 
of photographic images generated through photography of a 
common subject under a plurality of photographic 

15 conditions; 

an area discriminating step that distinguishes a 
background area of the photographic image from other areas 
in accordance with the photographic images obtained by the 
image obtaining step; 

20 a mixing state presuming step that presumes a 

mixing ratio of a subject color and a background color, and 
the subject color, in other areas excepting the background 
area in the photographic image, in accordance with the 
plurality of photographic images; and 

25 a background replacing step that replaces a 

background of the photographic image obtained by the image 
obtaining step with a different background using the 
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subject color and the mixing ratio, which are presumed by 
the mixing state presuming step. 
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